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ABSTRACT 

The paper describes the Shared Process Evaluation 
System (SHAPES) for analyzing the process of communHy development. 
The system's basic organizing unit is the critical incident, an event 
judged to be essential to the continuation of a particular community 
development project or as representing a choice point in the 
project's history. Three sets of design criteria were used in 
developing SHAPES: criteria related to information discrimination and 
gathering; those related to validity, bias, and reliability; and 
those related to flexibility of information storage and retrieval. In 
the system, .community development is defined as taking place when the 
priorities of different fields (individual or group actors in the 
community) are in relative agreement and result in effective shared 
activity. The SHAPE System in operation addresses itself to four 
questions: Hho was involved? What happened? In what contextual model 
should the project outcome be viewed? What were the project outcomes? 
These four questions together with the instruments designed to 
provide data for their analysis constitute the essence of SHAPES. A 
field test of SHAPES produced positive reactions from an outside 
evaluator, field agents, and community informants, thereby 
encouraging its further refinement and application. (JR) 
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Towards a Process Analysis of Community Development 



"There was Jio 'One, two, three and awayl' 
but they began running when they liked, and 
left off when they liked, so that it was 
not easy to know when the race was over." 

(from The Caucus Race in 
Al Ice in Wonderland by Lewis Carrol 
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INTRODUCTION AND RATIONALE 



We began our Investigation Into community development by asking 
ourselves If such a process or phenomenon really existed; and 
If It existed, what might It look like, what should we pay 
attention to, and what a field worker needed to knov/ to affect 
the process. 

Like the blind men examining an elephant, we found that we 
could study the process from several points of view, each leading 
to a different Interpretation '^f what actually happened within 
the community. If we used a powerful objective on our micro- 
scope we could differentiate the detailed Interpersonal structures 
within thto ..sDmmunlty; If we reversed the objective, we could see 
the entire community at a glance, but the fine details disappeared. 

We chose, therefore, to learn about community development by 
st'idylng both the whole-ness and part-ness of the phenomenon and 
ihe community. To do this we developed an evaluation system, 
the Shared Process Evaluation System (SHAPES^ which can be used 
to examine the process of community development at a variety of 
levels and for a variety of different purposes. The SHAPE System 
has these components: 

(a) It attempts to clarify the phenomenon In terms of Its overall 
shapes or patterns as a whole process. 

(b) It attempts to view the details of that process as parts of 
the whole and to view them in relation to each other and to the 
whole pattern. 

(c) It provides a means for sifting existing community development 
models with a view to assessing the relative strengths and weak- 
nesses of the models, to testing each one^s goodness of fit 
congruency with reality, and to aid in the construction of a 
testable composite model. 

(d) It can be used to examine the entire project and/or process 
or any part of that project and/or process. 

(e) It can be used to collect data while the project Is still 
In progress without disrupting community relationships, or to 
evaluate the project post facto. 

(f) It can be used by community people to develop a better 
understanding of the experiences they are sharing; by the field 
worker to provide diagnostic data on which to base Informed 
intervention strategies and process decisions; or by community 
development administrators anxious to determine relationships 
between process and products, between budgetary costs and 
outcome benefits. 
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THE FOCUS VARIABLE 



The Argument for a Smallest 
Meaningful Unit of Evaluation 



Much of the structure of the SHAPE System is founded on a new concept 
which we call the Smallest Meaningful Unit of Evaluation or SMU for 
short. At the basis of this concept is the assumption that a major 
function of any evaluation is the ability to foster ci.ear, concise, 
meaningful communication among a number of users of the system. Not 
only must evauative decisions be made, but trust must be generated 
among those to whom the project is important that the decisions have 
been made on the basis of a reality to which they can agree. 

Part of the difficulty in communicating about evaluation is that 
different people seem to organize information about the world at 
different .levels. An administrator of a federu: agency may see all 
programs under his control in terms of six to eight major program 
objectives. His decisions to redeploy resources are based on data 
organized around those objectives. A community development agent 
working in a community organizes his activities around local goals. 
This agent may see a series of meetings as falling into a project or 
a meaningful whole. On the other hand a citizen may see a single 
meeting or encounter with the agent as a meaningful unit. 

It is, of course, possible to conduct an evaluation at any level of 
complexity or at any level or organization of meaningful activity. 
The PPBS (Program Planning and Budgeting System) argues that 
evaluations should be conducted to ascertain the progress of a 
program toward basic agency goals and objectives. At program levels 
these basic goals are translated into specific activities. 

On the other hand, an evaluation can be organized at a micro level. 
During the past years there has been a movement to encourage program 
planners to be more specific about their program goals. In particular. 
It has been suggested tl^t program goals be broken down into small 
parts where specific behavioral, outcomes may be sought and expected. 
These behavioral objectives then form the building blocks of a 
program. An evaluation at the level of the behavioral objective unit, 
can be said to be at a micro level of analysis. 

The problem is not that several levels exist. Indeed, any comprehensive 
evaluation will include analysis at a variety of levels because the 
evaluation will serve a variety of audiences and purposes. The 
basic problem is which level should be selected as the focus of an 
evaluation system . The difference here is the selection of a level 
to conduct a single evaluation and the selection of a level to 
collect information on an ongoing basis. We argue that an evaluation 
system should focus on a level of evaluation that is meaningful to 
the largest number of audiences. If the evaluation level is seen as 
meaningful by these audiences; then, we predict there will be less 
resistance to the collection of data, there wll I be greater 
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confidence In the validity and reliability of the data and, 
consequently, there will be greater use made of the evaluative 
data. 

Experience in working with the users of evaluation systems indicates 
that, when there is a mismatch between the data gathering require- 
ments of an evaluation system in which the agents view their work, 
resistance to the evaluation system is hfgh. This is true both for 
systems in which the basic unit has been tho behavioral objective 
and for systems where the goals have been viewed as too global. 

For micro systems resistance has been related to the creation of large 
numbers of objectives. The problem has seemed to be that, after the 
specific objectives were created, it was too easy to lose sight of 
the overall pattern of the program. In other words, the field worker 
had been asked to divide tho program into unnecessarily small units. 
Problems at both levels can arise in a system such as the Extension 
Service's Extension Management System, Line items or specific tasks 
can become too small resulting in a proliferation of objectives for 
the field worker which are difficult to keep track of during the 
year. On the other hand, national objectives are often too large 
resulting in a difficulty in classifying certain events. The basic 
unit around which information is organized simply must correspond 
to a unit that the field worker finds meaningful. 

For the field worker, as well as the program administrators, there 
are probably several levels which are meaningful. How, then, can 
we choose that level which is most useful as the foundation of an 
evaluation system? It is clear that the smaller the unit the more 
precision which can result from the evaluation. Thus, the criterion 
for the basic unit may be expressed as that unit which is as small as 
possible without losing meaning to program personnel. This unit we 
call the Smallest Meaningful Unit, 

The Critical Incident as the Smallest 
Meaningful Unit of the SHAPE SYSTEM 

The basic organizing unit of the SHAPE System is the Critical 
Incident, The Critical Incident has many of the ideal characteris- 
tics of the Smallest Meaningful Unit discussed above. An incident is 
defined as an observable human activity which is sufficiently well 
differentiated to permit description and inferences to be made by 
participants and observers. To be a Critical Incident, the event 
must be judged as essential to the continuation of the project or 
as representing a choice point in the history of the project. The 
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participant or observer should be able to describe what happened 
before the incident, what occurred during the incident, what the 
outcomes or products were, and what happened following the incident. 
As a result of the definition, the Critical Incidents and the patterns 
which they form are viewed as meaningful by both community participants 
and community development professionals, these incidents can be 
analyzed in a way that is meaningful to program administrators. Thus, 
the Critical Incident meets the basic requirements of .the Smallest 
Meaningful Unit. 

Each Critical Incident contains an activity component. In our short 
experience with eliciting Critical Incidents we find that they tend 
to be seen as either meetings, definable tasks, or significant 
learnings. Thus, for each Critical Incident we identify the resources 
consumed during the incident, as well as results in the form of 
specific actions, decisions, products, or learnings. Learning can 
be defined in this setting as either the acquisition of additional 
facts or the reorganization of information already possessed. 

In addition to inputs and outputs we assist the participants in a 
Critical Incident to set the event in a perspective or overview of 
process. In order to minimize bias we assist this judgement with 
instruments, such as Q^sorts, and structured interviews. Thus, to 
each Critical Incident is attached input data, output data, activity 
data, participation data, and conceptual meaning data. 

Many of the questions which we wish to ask of a community development 
program relate to tie history or process which took place. The 
Critical Incident forms the building block of the analysis, but it 
is important that these units can be linked together. Each Critical 
Incident must have identifiable predecessors and successors. 
Patterns formed by the history of Critical Incidents allows classi- 
fications of patterns for macro analysis, while the unit itself 
yields smaller units of information for more micro analysis. 

One additional property of the use of Critical Incidents is worth 
noting. In the SHAPE System we ask each individual informant (Field) 
to provide us with a list of Critical Incidents as perceived by the 
individual. The basic datum is a Critical Incident as defined by one 
individual. As we accumulate these individual Critical Incidents we 
begin to be able to determine the degree to which certain events are 
perceived by many individuals as Critical Incidents. Thus, we can 
determine the degree of congruence among the participants in a 
particular project. Those Critical Incidents which are identified 
by many Individuals become the marker points in describing and 
evaluating what happened during a particular project. 
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From our experience with community development programs, as well 
as extensive discussions with other community development pro- 
fessionals, we have developed three sets of design criteria which 
we used In developing the SHAPE system. They are: criteria 
related to Information discrimination and gathering; criteria 
related to validity, bias, and reliability; and criteria related 
to flexibility of Information storage and retrieval. 

Criteria Related to Information Discrimination and Gathering . 

We believe that an evaluation system which seeks to provide data for 
decisions relating to community development programs should: 

(a) focus on the process, structures, and the observable 
outputs of a community development activity; 

(b) be readily usable by either an ^'outside" or an 
"Inside" evaluator; 

(c) require minimal skills likely to be found in community 
development field professionals; 

(d) either be nonreactive or support program goals during 
the collection of data. 



Criteria Related to Validity, Bias, and Reliability 

Any system which provides evaluative data should possess character- 
istics of validity, freedom from bias, and reliability. In a system 
which collects data from community development activities these 
characteristics are difficult to attain. Nevertheless, we feel 
that the following criteria should apply: 

(a) the data gathered should be Judged by the users of the 
system as fairly representing reality; 

(b) the data should provide an accurate documentation of the 
community development process for purposes of assessment 
and reporting; 

(c) the data should be as free as possible from the bias of 
the community development field professional; and 

(d) the data collected should be rep I! cable by other data 
col lectors. 
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Criteria Related to TIexibI I Ity of 
Information Storage and Retrieval 

We envision a large number of users of the evaluation system. Each 
of these users will be asking for data to assist In the making of 
diverse decisions about the ccxnmunlty development program. In light 
of these needs we decided to attempt to design a system which 
allowed data to be retrieved In flexltile formats. The following 
criteria were developed to meet this goal: 

(a) the system should provide data which allows comparison 
among community development programs, even If the 
programs were designed from different conceptual 
bases and with different goals; and 

(b) the system should accommodate different models 
of community development. 



The SHAPE System 

Using our notion of the Smallest Meaningful Unit and the list of the 
criteria developed in the last section, we set out to design a system 
of data collection which could be used In evaluating community 
development programs. We refer to this effort as a system because 
we were trying not only to provide help with the kind of variables 
which might be considered, but we also wanted to develop the 
instruments which would be usable and effective In the field. 



Three Necessary Assumptions 

We have made an attempt to reduce the number of assumptions necessary 
yo udr our system to an absolute'minlmum. The primary reason for the 
attempt tp minimize necessary assmptlons is to permit the system to 
be used by as many Individuals with varying conceptual frameworks as 
possible. The extent to which we have succeeded in reducing the 
assumptions will be left to our critics, but at this point we want 
to share the necessary assumptions. 

Assumption 1. Human activity related to need fulfillment 
can be described and broken Into phases. 

We do not know if this assumption is correct, but we are In good company 
Almost all of the theoretical writings in community development have 
assumed that regularities of some kind exist In the activities of 
individuals involved in a problem solving process. If the assumption is 
false, then there can be no way of measuring or accumulating the results 
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Assumption 2. The human activity related to need fulfillment 
takes place in a number of Fields (consisting 
of individuals or groups) which have specific 
describable characteristics. 

Any Field can and does operate independently, but when several Fields 
come into contact a potential for shared activity exists. Shared 
activity has the potential for increasing or amplifying the effect 
of an individual Field,'s efforts. 

Assumption 3. The achievement of the potential for shared 

activity depends on a match of phases between 
differing Fields. 

We believe that effective shared activity occurs when two or more Fields 
act together and view the action as serving the same functions. Each 
individual Field's agenda serves that Field as a plan complete with 
expectations, intentions, and attentions. When there is a congruence 
between the agendae of several Fields, there is effective shared 
activity. In other words, the Fields are now striving for similar 
goals in relative harmony. 



Definition of Community Development 

Taking these assumptions together, we are now ready to define 
community development in terms of ie SHAPE System. Community 
Development takes place when the priorities of different fields 
are in relative agreement which results in effective shared 
activity. 



Four Basic Questions Attended to by the SHAPE System 



The SHAPE System contains no new lines of inquiry, but it does bring 
together three traditions in the research of community education. The 
basic questions of the system, the research traditions from which we 
drew the questions and our initrumentat ion are presented in Figure 1. 

The first questions is: Who is involved? In many studies this question 
is pursued by asking community merpbers to nominate a list of people 
who qualify as community leaders. By combining the nominations from 
a large number of informants one is able to accurately ascertain who 
the perceived leaders in a community are. Although the SHAPE System 
basically use^ this strategy, we have made a major conceptual 
modification. Instead of collecting individuals, we view those who 
were involved as operating in a subpart or Field of the community. 



1. Ronald C. Powers, *^Power actors and social change (Parts 1 and 11) 
Journal of Cooperative Extension , 1967, vol. 5, pp 153-163 and 238-248. 

2. Terry Patterson, "Spatial and temporal analysis of group functions: 
A categorization system for analysis of community activities". Unpublished 
manuscript. (Toronto, Ontario: Addiction Research Foundation, 1974, 
Project H 130; Subsidy No. 603), 
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The problem Is that In a community activity people do not always act 
as individuals, A person can be acting as an individuals, as a part 
of a small croup, or as a representative of an agency or institution. 
There are four classifications of Fields: (1) individuals; 
(2) small group; (3) institutions; or (4) community. We collect 
data pertaining not only to who was involved, but also under what 
conditions they were involved. Were they constrained by policy or 
were they free to act as individuals? Did they carry a particular 
community role, or were they a member of an identifiable s;.;al I group? 

The second major question which the SHAPE system addresses is: 
What happened during a particular community developmentproject? In 
order to collect and organiz^ these data we have adopted Flanagan's 
Critical incident Technique. The Critical Incidents form the focus 
variable of the SHAPE System. Around the Critical Incident we 
collect all of the remaining data. 

We ask Individuals to think through the history of the project or 
series of events, and isolate those events which seem to have been 
critical. By critical, we mean events which either mark a turning 
or choice point in the project or events which were necessary for the 
continuation of the project. We ask these questions both of the 
community development professionals and of individuals who have been 
identified in the irientif ication of Fields. We combine the Critical 
Incidents gathered fron these several sources into a history or 
chronology of the project. 

The third question of the SHAPE System is in what context should we 
view whan had happened? There are many possible models that one 
could propose as the context. We have chosen a six step model pro- 
posed by one of the authors of the SHAPE System. With proper 
instrument modification it would be possible to substitute any model 
of community development. The only requirements are that the model 
describe reality and the phases be capable of translation into 
activities which can be differentiated by the participants. 



3. John C. Flanagan, Measuring Human Performance , 
(University of Pittsburgh and American Institutes for Research, 
1962). 

4. Jerry Patterson, Op cit . 
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There are many ways that one could match the Critical Incidents with 
various model phases. Perhaps the simplest (and most open to bias) 
would simply be to present to the informant a definition of the 
various phases and ask him to Judge into which phase the particular 
event should be categorized. Alternately, the interviewer could 
interview the informant and subsequently make the judgment. Finally, 
one could develop a series of instruments which attempt to match 
specific behaviors with phases and then ask the informant to select 
these behaviors which characterize the event and then infer the 
jDhase from the data collected through the instrument. It is this 
final strategy which forms the bulk of the SHAPE System. We have 
developed a series of instruments to help control bias: a probe 
Sheet, a Q-sort, and a set of Phase Description Cards. 

As we interview each individual about the Critical Incidents, we use 
various instruments to identify the meaning each incident had for 
that individual. Thus, we have a phase determination for each 
individual for each Critical Incident and these form the plots on 
our SHAPES analysis. 

It i£> necessary to collect this information from a variety of 
informants because it is possible for each of the people attending 
a particular event to attach a different meaning to the event. 
Indeed, it is the degree of congruence between meanings attached 
to a particular event that we are interested in determininp. 

As an example, a community worker may attend a meeting and 
characterize the meeting as a time of identifyinp community needs. 
For a man who represented the state planning office, the meeting 
may have been seen as an opportunity to carry out the activities 
of his agency by informing the community concerning the services 
available from his agency. These two people have attended the same 
meeting, but it has had a different meaning to each and each may 
evaluate the outcomes differently. 

At the same time as we are focusing on the three questions above, 
we are also interested in collecting data on the outcomes of the 
project. For each Critical Incident we asked: What happened? 
What resulted from the event? How is the community different? 
These data are necessary to assess the degree to which the project 
is meeting its goals. But notice, this outcome data is collected 
as a part of focusing on the process of community development, and 
thus is net isolated. We are primarily interested in what is 
happening in the community and only secondarily interested in the 
specific goals of the agency supplying the community worker. The 
outcome data are necessary, but not sufficient in themselves. 



ERLC 



012 



These four questions, and the Instruments designed to provide data 
for their analysis, form the SHAPE System. The handbook^ setting 
forth the system Indicates that we have Indeed designed a community 
development evaluation system which meets to a large extent the 
criteria set forth above. 

We next turn to a short description of ogr experience in field 
testing the SHAPE System. 



Experience with the SHAPE System 



We would like to share some of the preliminary experiences we have 
had in testing the SHAPE System. Although the development of the 
system is still continuing, we have had several opportunities to 
field test both the concept and the Instruments which have been 
developed. The most complete field test to date was conducted in 
Williams, Arizona, in December of 1974. The basic analysis is 
available from the authors, but for this paper we would like to 
share some of the reactions we received from the outside evaluator, 
the field agents, and the community informants. 

Although we were not able to obtain complete data because several 
informants were out of town, we were able to collect most of the 
necessary data in only a day and a half of interviewing. It is^ 
anticipated that most community projects could be evaluated in less 
than a week. 



The reaction of the field agents to the experience was qult^-^ 
positive. Prior to the field test there was a natural reluctance 
on the part of the agen^s to the use of an outside evaluation 
system. After working with SHAPES the agents reported that they 
found both the process and the followup Informative and useful in 
their programming efforts in the community. The resulting data 
was described as fairly representing the reality of the community 
development project. We were able to provide clear evidence of a 
correlation of extension efforts and outputs resulting from the 
process. 



5. Lynn Davie, Terry Patterson, Dorothy MacKeracher and 
Rlcho.-d Cawley, "SHAPES: Shared Process Evaluation System" 
A project funded by the Extension Service, U.S. Department of 
Agriculture. Handbook available from the Department of Adult 
Education, Ontario Institute for Studies in Education, Toronto, 
Ontario, Apri I 1975. 



e. Lynn Davie et al , "SHAPES: Case Study". Copies available 
from The Department of Adult Education, O.LS.E., Toronto, Ontario, 
December 1974. 
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One of the difficulties of evaluating a community davelopment 
project is that it is seldom possible to link an output with a 
specific input by the community development professional. Indeed, 
often the strategy of the professional is to play a facilitative 
or catalytic role and to avoid making interventions that lead to 
specific community outputs. The SHAPE System allows the presen- 
tation of a correlation of effort with the increased movement and 
productivity of a community. These patterns are often stable and 
clearly identifiable when utilizing the SHAPE System. 

In addition, we were able to test the notion that field agents 
would be able to use the SHAPE System with a short, but well 
organized, training program. The instruments and concepts were 
extensive, but not too complex to understand. Within a day we 
were able to train the agents to use the basic interview strategies 
and instruments. 

The community participants responded well to the interviews. We 
were weli received, and the participants were most willing to talk 
to us. They did not seem to have difficulty with any of the basic 
questions, though the Q-sort was difficult for some of the less 
sophisticated individuals. This limitation did not cause great 
difficulty as we had alternative means to assess the particular 
phase of a Critical Incident. One finding which we found most 
interesting is that many individuals in the community were not 
aware that a project or process was taking place in their community. 
Many of the individuals with whom we talked viewed the various 
activities as isolated or unrelated events. This finding surprised 
the agents involved but did not limit the ability of the informant 
to provide us with the information we needed. 

Our overall impression of reactions to the SHAPE System is that it 
deals with information at a level that many found meaningful and, 
at the same time, provides us with the basic patterns needed for more 
sophisticated sna lysis. We did not feel that we in any way dis- 
rupted or distorted the relationship between the agents and the 
community leaders wutg whom they were working. On the contrary, we 
were able to identify several places where new effort might be 
productively directed. In addition, we were able to show the clear 
effects of the extension effort in a project where those effects 
were being claimed by other agencies. We are most encouraged by 
this field test. 

Over the next few months we will be testing the system in other 
extension efforts in the Western Region of the United States, and 
with a number of community development programs in Canada. 
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When we set out to learn about community development we tried 
not to impose our own biases on the data we gathered. We did 
make some basic assumptions: that good community devetment 
Involves shared activities, shared understandings, and shared 
perceptions; and that this sharing process occurs among a 
diverse collection of Individuals and groups. We therefore 
selected three variables to examine more closely: who was 
involved, what actually happened, and what meanings were 
attached to what happened by those Involved. We then attempted 
to relate these three variables in a manner which yielded a 
three-dimensional picture of the phenomenon occurring in that 
community. 

The uses to which this system can be put have yet to be fully 
explored. They can vary according to the needs of the person 
doing the looking and taking the picture. Whatever use one 
miglit have for the 1 nformation, we believe there is ii.erlt in 
using a process which focuses on the means of community 
development as well as on the ends and in using a shared 
perspective based on actual events. 

USES OF THE SHAPE SYSTEM 



We hope the SHAPE System will be used both in further community 
development field tests and in other settings. In the SHAPES 
Handbook we have suggested three^ways in which the System might 
be used by community developers. It is also our conviction 
that the system could be adapted to test other models of the 
community development process. This would require changes in 
the phase descriptions and corresponding changes in the phase 
designation instruments. 

We are also looking forward to adapting the SHAPE System for use 
in other settings, such as the teaching- 1 earning process or in 
the professional development process. In these adaptations the 
instrumentation would need to be changed more radical ly but the 
concepts involved (e.g. the smallest meaningful unit and the 
shared aspects of the process) would be retained. 

Wha-*-ever use the reader selects, we would be happy to be kept 
informed of progress made and results obtained, and of 
reactions to the system ch 'o v-orking tool. Please forward such 
comments and/or reports to Dr. Lynn Davie, fjopartmenT of Adult 
Education, The Ontario Institute for Studies in Education, 
252 Bloor Street West, Toronto, Ontario. 



7. Lynn Davie et al . Op. clt. 
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